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Abstract 


The Interval Management for Near-term Operations Validation of Acceptability (IM-NOVA) 
experiment was conducted at the National Aeronautics and Space Administration (NASA) 
Langley Research Center (LaRC) in support of the NASA Airspace Systems Program’s Air 
Traffic Management Technology Demonstration- 1 (ATD-1). ATD-1 is intended to showcase an 
integrated set of technologies that provide an efficient arrival solution for managing aircraft 
using Next Generation Air Transportation System (NextGen) surveillance, navigation, 
procedures, and automation for both airborne and ground-based systems. The goal of the IM- 
NOVA experiment was to assess if procedures outlined by the ATD-1 Concept of Operations 
were acceptable to and feasible for use by flight crews in a voice communications environment 
when used with a minimum set of Flight Deck-based Interval Management (FIM) equipment and 
a prototype crew interface. To investigate an integrated arrival solution using ground-based air 
traffic control tools and aircraft Automatic Dependent Surveillance-Broadcast (ADS-B) tools, 
the LaRC FIM system and the Traffic Management Advisor with Terminal Metering and 
Controller Managed Spacing tools developed at the NASA Ames Research Center (ARC) were 
integrated into LaRC’s Air Traffic Operations Laboratory (ATOL). Data were collected from 10 
crews of current 757/767 pilots asked to fly a high-fidelity, fixed-based simulator during 
scenarios conducted within an airspace environment modeled on the Dallas-Fort Worth (DFW) 
Terminal Radar Approach Control area. The aircraft simulator was equipped with the Airborne 
Spacing for Terminal Area Routes (ASTAR) algorithm and a FIM crew interface consisting of 
electronic flight bags and ADS-B guidance displays. Researchers used “pseudo-pilot” stations to 
control 24 simulated aircraft that provided multiple air traffic flows into the DFW International 
Airport, and recently retired DFW air traffic controllers served as confederate Center, Feeder, 
Final, and Tower controllers. Analyses of qualitative data revealed that the procedures used by 
flight crews to receive and execute interval management (IM) clearances in a voice 
communications environment were logical, easy to follow, did not contain any missing or 
extraneous steps, and required the use of an acceptable workload level. The majority of the pilot 
participants found the IM concept, in addition to the proposed FIM crew procedures, to be 
acceptable and indicated that the ATD-1 procedures could be successfully executed in a near- 
term NextGen environment. Analyses of quantitative data revealed that the proposed procedures 
were feasible for use by flight crews in a voice communications environment. The delivery 
accuracy at the achieve-by point was within ±5 sec, and the delivery precision was less than 5 
sec. Furthermore, FIM speed commands occurred at a rate of less than one per minute, and pilots 
found the frequency of the speed commands to be acceptable at all times throughout the 
experiment scenarios. 
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1 Introduction 

Air traffic demand is predicted to increase by 2 to 3 percent per year over the next 20 years, with 
the number of revenue passenger miles nearly doubling by 2032 (ref. 1). If the current air 
transportation system is left unmodified, this projected growth will lead to increased delays, fuel 
costs, noise pollution, and greenhouse gas emissions. The Federal Aviation Administration’s 
(FAA) Next Generation Air Transportation System (NextGen) concept envisions a 
comprehensive transformation of the National Airspace System (NAS) to support this continued 
growth in a safe, reliable, and efficient manner (ref. 2). The National Aeronautics and Space 
Administration (NASA) is collaborating with the FAA and other industry partners to develop 
advanced technologies and automation tools necessary for NextGen. 

Improving the efficiency of terminal area arrival operations is an especially complex task. 
Conditions in busy terminal areas today often result in inefficient arrival paths involving frequent 
changes in speed, heading, and altitude to maintain safe separation between aircraft and absorb 
large amounts of delay. These inefficiencies lead to increased fuel burn and noise pollution, as 
well as higher controller workload and traffic congestion. Furthermore, greater uncertainty in the 
current system causes controllers to add separation buffers between aircraft, thus reducing 
throughput and increasing delays. Although more efficient arrivals are available, current 
technology limits their use to periods of light to moderate traffic conditions. New concepts and 
technologies are needed to make efficient arrival procedures feasible during heavy traffic. 

NASA’s Air Traffic Management Technology Demonstration- 1 (ATD-1) will operationally 
demonstrate the feasibility of efficient arrival operations by combining ground-based and 
airborne NASA technologies (refs. 3, 4, and 5). The ATD-1 integrated system consists of the 
following three core components: 

• Traffic Management Advisor with Terminal Metering (TMA-TM), which generates 
precise time-based schedules to the runway and merge points within the terminal area 

• Controller-managed spacing (CMS) decision support tools, which provide controllers with 
speed advisories and other information needed to meet the schedule 

• Flight Deck-based Interval Management (FIM) avionics and procedures, which allow 
flight crews to adjust their speed to achieve precise relative spacing 

The Traffic Management Advisor (TMA) was originally developed at NASA Ames Research 
Center (ARC) and is currently used at Air Route Traffic Control Centers nationwide to determine 
an appropriate arrival schedule (ref. 6). TMA-TM is an enhanced form of TMA that includes 
terminal area metering and enables the use of more efficient arrival procedures. CMS decision 
support tools were also developed at NASA ARC. They provide controllers with the information 
necessary to achieve arrival schedule conformance using speed commands, thus reducing the use 
of tactical vectoring (refs. 7 and 8). The use of TMA-TM in conjunction with CMS tools has 
been assessed, and results indicate an increase in airport throughput (refs. 9-12). 

FIM is an airborne spacing concept in which the flight crew is responsible for flying their aircraft 
at a speed that achieves their assigned time-based spacing interval behind a target aircraft, while 
Air Traffic Control (ATC) remains responsible for ensuring that all aircraft maintain safe 
separation. Typically, ATC designates a spacing buffer in addition to the separation requirement 
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to ensure that separation is always maintained. The goal of airborne spacing is to decrease this 
spacing buffer by decreasing the variability of the time error associated with an aircraft’s arrival 
at a specific point along its arrival route. The precise merging and spacing enabled by FIM 
avionics and flight crew procedures reduces excess spacing buffers and results in higher terminal 
throughput. Studies conducted by MITRE (refs. 13-15), EUROCONTROL (refs. 16-19), and 
NASA Langley Research Center (LaRC) (refs. 20-22) have demonstrated an increase in 
efficiency through the use of FIM operations. 

In addition to utilizing these advanced technologies, aircraft will fly new, more direct area 
navigation (RNAV) routes that extend from en route airspace to the runway. Optimized profile 
descent (OPD) procedures will also be implemented to provide a fuel-efficient continuous 
descent approach rather than the step-down descents used today. The Automatic Dependent 
Surveillance-Broadcast (ADS-B) infrastructure currently being implemented by the FAA will 
also be leveraged. The FIM tools will calculate speed commands using information provided by 
ADS-B, which is more accurate than traditional radar. The ability of flight crews to make more 
precise speed adjustments will enable a reduction in spacing buffers resulting in higher terminal 
throughput. 

These technology components and procedures have been evaluated independently, and each has 
demonstrated benefits. As an integrated system, these technologies are intended to increase 
throughput, reduce delay, and minimize environmental impacts. Initial studies at NASA ARC to 
demonstrate the ATD-1 concept and validate operational feasibility indicate that the concept is 
viable and operations are acceptable (refs. 23-25). 

1.1 Current Study 

As part of the preparations for an ATD-1 flight demonstration, the Interval Management for 
Near-Term Operations Validation of Acceptability (IM-NOVA) human-in-the-loop (HITL) 
experiment was conducted at NASA LaRC. The objective of this experiment was to assess if the 
procedures outlined in the ATD-1 Concept of Operations (ConOps) document (ref. 5), when 
used with the integrated ATD-1 technologies, were acceptable to and feasible for use by flight 
crews in a voice communications environment when precision to an achieve-by point (i.e., the 
final approach fix) was expected. This paper describes the experiment’s methodology, results 
associated with the pilot participants’ acceptability and workload ratings, and results of the 
evaluation of the flight crew procedures’ feasibility. 
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2 Method 

2.1 Experiment and Scenario Design 

Data were collected from current 757/767 pilot participants asked to fly a high-fidelity, fixed- 
base simulator during scenarios conducted within an airspace environment modeled after the 
Dallas-Fort Worth (DFW) terminal radar approach control (TRACON) area. Each experiment 
scenario consisted of multiple air traffic flows involving 25 arrival aircraft flying into DFW 
airport and landing on runways 17C and 18R. All aircraft flew RNAY arrivals to the Instrument 
Landing System (ILS) approaches. Some aircraft initialized in level cruise and flew the full 
arrival and approach to the runway, whereas others initialized in descent and flew only a portion 
of the arrival before flying the approach. One of the arrival aircraft in the arrival stream was a 
high-fidelity, fixed-base simulator with pilot participants operating as a two-person crew. This 
simulator was equipped with NASA LaRC’s airborne spacing algorithm, Airborne Spacing for 
Terminal Area Routes (ASTAR) (version 11.06.22) (ref. 26), and a prototype FIM crew interface 
(shown in Figs. Figure 1 and Figure 2). The remaining 24 arrival aircraft were flown by two 
researcher pseudo-pilots, each of whom used a graphical user interface (GUI) to control multiple 
medium-fidelity aircraft simulators. To provide a realistic traffic environment, each scenario also 
included 25 departure aircraft. Recently retired DFW air traffic controllers served as confederate 
Center, Feeder, Final, and Tower controllers issuing speed commands, vectors, and IM 
clearances. 

When performing FIM operations, pilot participants were expected to use ASTAR-provided 
speed guidance whenever possible. This speed guidance is designed such that the FIM, or 
spacing, aircraft will achieve the assigned spacing goal (ASG) behind its target, or lead, aircraft 
at a predefined achieve-by point while remaining within 10 percent of the published RNAY 
arrival airspeed. In this experiment, the final approach fix (FAF) served as the FIM aircraft’s 
achieve-by point. The prototype FIM crew interface shown in Figure 1 and Figure 2 consisted of 
two side-mounted electronic flight bags (EFBs) and two ADS-B guidance displays (AGDs) 
mounted under the glare shield within the flight crew’s forward field-of-view. Figure 1 shows 
the position of a side-mounted EFB as well as a screen shot of the FIM application used for data 
entry and speed conformance monitoring. Figure 2 shows the position of an AGD as well as an 
image of the AGD displaying the IM-commanded speed and the aircraft’s deviation from the 
commanded speed. 
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Figure 1. Side-mounted electronic flight bag (EFB) and Flight Deck-based Interval 

Management (FIM) application. 



Figure 2. ADS-B guidance display (AGD) presenting target and error speed 

values. 

Previous FIM research conducted at NASA LaRC has utilized data link to transfer information 
from ATC to the flight crew. However, since the ground infrastructure necessary to support data 
link will not be available during the execution of near-term FIM operations, voice 
communications will be relied upon to transfer necessary information. In the IM-NOVA 
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experiment, confederate controllers issued IM clearances to the flight crews, who then entered 
information into the EFBs, and activated the FIM avionics. The IM procedure required the flight 
crew to enter the following pieces of information included in the IM clearance into the EFBs: 

• IM achieve-by point (i.e., FAF) 

• Scheduled time of arrival at the IM achieve-by point 

• Target aircraft callsign 

• ASG (spacing interval required at the IM achieve-by point) 

• Target aircraft flight path (arrival and transition) 

Five flight scenarios were defined using the 1x5 experiment matrix shown in Table 1 to allow an 
examination of the five flight crew procedures outlined in the ATD-1 ConOps document. 
Additional details regarding the design of the experiment’s scenarios and graphical 
representations of the research standard terminal arrival routes (STARs) used within the 
experiment are included in Appendix A. 


Flight Crew 
Procedure 


Table 1. Experiment design matrix. 


Nominal 

Amend 

ASG 

ATC 

Termination 
of FIM 

Suspend/Resume 

FIM 

ADS-B 

Loss 


1. The Nominal scenario consisted of an IM clearance issued by ATC prior to top of descent 
(TOD). After the FIM aircraft reported that spacing had commenced, the pilot 
participants’ aircraft (i.e., the FIM aircraft) maintained nominal FIM operations until 
reaching the achieve-by point. 

2. During the Amend ASG scenario, the initial IM clearance was issued shortly after TOD. 
Approximately two minutes after the FIM aircraft reported that spacing had commenced, 
ATC issued an amended clearance to increase spacing by 20 sec. 

3. In the ATC Termination of FIM scenario, the initial IM clearance was issued shortly after 
TOD. Once the FIM aircraft and target aircraft were inside the DFW TRACON, ATC 
cancelled the IM clearance and vectored the target aircraft for landing on runway 13R. 
ATC then issued a new IM clearance with a new target for the FIM aircraft. 

4. The Suspend/Resume FIM scenario consisted of an IM clearance issued by ATC prior to 
TOD. After the FIM aircraft reported that spacing had commenced, ATC suspended the 
IM clearance and issued a speed change of 20 kt for the FIM aircraft. Approximately two 
minutes later, ATC cleared the FIM aircraft to resume IM spacing. 

5. During the ADS-B Loss scenario, ATC issued the initial IM clearance prior to TOD. 

After the FIM aircraft reported that spacing had commenced and both the pilot 
participants’ aircraft and target aircraft were inside the TRACON, the target aircraft 
experienced a loss of ADS-B capability. The pilot participants notified ATC that they 
were “IM Unable” due to a loss of the target aircraft’s ADS-B signal, and ATC then 
cancelled the initial IM clearance and issued a new clearance with a new target aircraft. 

During all five scenarios, the closest aircraft in the arrival stream for the same runway as the FIM 
aircraft was designated as the initial target aircraft. If a new target was designated later in the 
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scenario, it was always the next closest aircraft in the arrival stream. For both the ATC 
Termination of FIM and ADS-B Loss scenarios, the second clearance was issued in the 
TRACON at an altitude below 10,000 ft. 

2. 1. 1 Hypotheses 

The IM-NOVA experiment was designed to meet the experiment objective and investigate the 
following a priori hypotheses: 

Hypothesis 1 : The use of the procedures for receiving and executing IM clearances in a voice 
communications environment will be acceptable to the flight crew, with mean ratings of the 
procedures’ completeness and acceptability higher than 4.5 on a 7-point scale. 

Hypothesis 2: At least 90 percent of the pilots will report that the procedures are complete. 

Hypothesis 3: Pilots will report the workload level required to execute the procedures to be 
acceptable, with a mean rating of less than 3 on the Modified Cooper-Harper (MCH) rating 
scale 1 (ref. 27). 

Hypothesis 4: Pilots will report no increase in workload with the Amend ASG, ATC Termination 
of FIM, Suspend/Resume FIM, or ADS-B Loss scenarios as compared to the Nominal scenario 
(i.e., a difference of less than 1 unit on the MCH rating scale). 

Hypothesis 5: Pilots will report that the crew interface was usable, with mean ratings of the 
acceptability of heads-down time required and usability of the displays higher than 4.5 on a 7- 
point scale. 

Hypothesis 6: The data entry procedures will be acceptable to the flight crew, with mean ratings 
of ease and intuitiveness of entering information higher than 4.5 on a 7-point scale. 

Hypothesis 7: The rate of speed commands will be acceptable to the flight crew, with speed 
commands occurring at a rate of less than two per minute, and mean ratings of operational 
acceptability and acceptability of the frequency of speed commands higher than 4.5 on a 7-point 
scale. 

Hypothesis 8: The spacing error at the FAF will be within ±5 sec with a standard deviation of 
less than 5 sec. 

2.2 Pilot Participants 

The study involved 20 current 757/767 pilots from major U.S. air carriers who participated in the 
experiment in 10 groups of two-person crews. All pilots were male and ranged in age from 40 to 
62 years. On average, each of the pilots had 23 years of airline experience and over 13,000 hours 
of commercial airline flight time. To minimize potential effects associated with different airline 
operating procedures, all two-person crews were paired from the same airline, and the pilots flew 
in their current operational position (captain or first officer) using their company’s standard 
operating procedures modified to include FIM operations. 


1 A rating of 3 on the MCH rating scale indicates that the instructed task is fair and/or has mild difficulty, and 
acceptable operator mental effort is required to attain adequate system performance. The full MCH rating scale is 
presented in Appendix C. 
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2.3 Experiment Procedure 

The pilot participants received training materials and had access to computer-based training prior 
to arriving at LaRC. After arriving at LaRC, pilot participants received four hours of classroom 
and hands-on training involving the completion of three simulated flight training scenarios prior 
to completing the first data collection run. Each two-person crew participated in a two-day 
experiment session and completed a total of 10 data collection flights. The first day began with 
training which was followed by data collection flights. Each data collection flight lasted 
approximately 25 min, and pilots were asked to complete a post-run questionnaire immediately 
following each flight. The second day consisted of the remaining data collection flights and post- 
run questionnaires, the completion of one exploratory scenario involving the presentation of 
aural cues in conjunction with commanded speed changes, and the completion of a post- 
experiment questionnaire and debriefing session. 

Every crew flew each experiment scenario twice: once with the captain as the pilot flying (PF) 
and the first officer as the pilot monitoring (PM), and once with the first officer as the PF and the 
captain as the PM. The run order of the scenarios was partially counterbalanced, and within each 
crew the pilots switched PF and PM responsibilities between runs. Additional details regarding 
the experiment run order are included in Appendix B. 

2.4 Scheduling and Spacing Technologies 

This experiment utilized an integrated set of ground-based and airborne technologies consisting 
of TMA-TM, CMS decision support tools, and FIM avionics and procedures. These scheduling 
and spacing technologies are described below. 

2.4. 1 Traffic Management Advisor with Terminal Metering (TMA-TM) 

TMA-TM is an extension of the operational TMA that determines an arrival schedule based on 
airport conditions, airport capacity, required spacing, and weather conditions. This scheduling 
tool calculates the estimated time of arrival (ETA) and the corresponding scheduled time of 
arrival (STA) at various meter and merge points along the aircraft flight path. The TMA-TM data 
are broadcast to the en route and TRACON controller positions for use by the CMS tools to 
assist the controllers in maintaining optimum flow rates to the runways. 

The TMA-TM display used during the IM-NOVA experiment (shown in Figure 3) included a 
total of six timelines. From left to right, the first four timelines show the TMA schedule for each 
of the four metering fixes around the DFW TRACON. The fifth timeline shows the arrival 
schedule for DFW runway 1 8R, and the sixth timeline shows the arrival schedule for runway 
17C. The current clock time (1541 :01z) appears in the upper left comer. The white numbers 
located in the middle of each timeline indicate minutes after the hour. As time progresses, the 
timelines scroll toward the bottom of the screen with the current time at the bottom and one hour 
in the future at the top. On the left of each timeline, each aircraft’s ETA at the applicable 
metering fix or runway threshold is shown in green. On the right side of the timeline, each 
aircraft’s STA is shown in amber or blue. An amber callsign indicates that an aircraft is outside 
of the TMA freeze horizon for the corresponding fix/runway (set at 19 minutes in the future). 
Until the freeze horizon is passed, TMA continuously recalculates the STA. When the freeze 
horizon is reached, the STA is frozen, and the aircraft callsign turns blue. If the green indicator 
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on the left side of the timeline is below the corresponding indicator on the right, the aircraft is 
early; if it is above the corresponding indicator, the aircraft is late. 
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Figure 3. Screenshot of the Traffic Management Advisor with Terminal Metering 
(TMA-TM) display used in the IM-NOVA experiment. 


2.4.2 Controller-Managed Spacing (CMS) 

During the IM-NOVA experiment, three CMS tools were used to provide the confederate 
controllers with information needed to meet the TMA-TM generated schedule: (1) early/late 
indicators, (2) slot marker circles, and (3) speed advisories. Early/late indicators located in the 
aircraft full data blocks (FDB) enabled controllers to quickly assess the schedule-conformance 
information for that aircraft. Elements included in the FDB shown on the Center controller’s 
display, including an early/late indicator, are depicted and described in Figure 4. 
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NASA9-3 
36GC 

233 465 <&S 
B 752 
E0 : 2 4 


Line 1 : Aircraft callsign 

Line 2: Aircraft altitude (in hundreds of feet) 

Line 3: 

Item 1 : Computer assigned aircraft identification 

number 

Item 2: Ground speed 

Item 3: Magenta symbols, manually toggled by ATC 
® = Aircraft’s STA has been assigned, but no IM 
clearance has been issued 

® S = IM clearance has been issued, but IM has not 
yet been engaged (i.e., the aircraft is still flying 
to its STA) 

S = Aircraft is operating in IM mode 
Line 4: Aircraft type (B752 = B757-200) 

Line 5: Early/Late indicator (shows that aircraft is 
currently 24 sec early) 


Figure 4. Elements in the aircraft full data block (FDB) shown on the IM-NOVA 

Center controller’s display. 


Figure 5 depicts and describes the elements of an FDB with a slot marker circle and a speed 
advisory as shown on the Feeder controller’s display. The IM-NOVA experiment’s confederate 
TRACON controllers’ displays presented slot marker circles that indicated where an aircraft 
should be located at a given time if it were to fly the RNAV arrival, meeting all published speed 
and altitude restrictions and meeting the STA at the next scheduling fix. The relative position of 
the aircraft symbol and the slot marker circle provided a quick visual indication of how an 
aircraft was positioned relative to its STA. The diameter of the slot marker circle represented 15 
sec of flying time at the aircraft’s current ground speed. If the aircraft was on schedule, the 
aircraft symbol was centered within the slot marker circle. 

Speed advisories in an aircraft’s FDB helped confederate controllers formulate speed clearances 
for aircraft not performing FIM operations. The speed advisory provided a recommended 
calibrated airspeed (CAS) intended to place an aircraft back on schedule before reaching a 
scheduling fix. 
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Line 1 : 

Item 1: Aircraft callsign 
Item 2: Assigned runway 
Line 2: 

Item 1 : Aircraft altitude (in hundreds of feet) 

Item 2: Ground speed 

Item 3: Aircraft class (F = 757) 

Item 4: S (in magenta) = Aircraft is operating 
in IM mode 

Line 3: Speed advisory (i.e., recommended 
CAS) 

Slot marker circle with aircraft symbol “H” 



Figure 5. Full data block (FDB) with speed advisory and slot marker circle as 
shown on the IM-NOVA Feeder controller’s display. 

2.4.3 Flight Deck-based Interval Management (FIM) 

The FIM tools provide onboard speed guidance to the flight crew, enabling them to achieve a 
precise spacing interval behind a target aircraft and thereby meet a schedule set by TMA-TM. To 
allow the performance of FIM operations during the IM-NOVA experiment, the simulator flown 
by the pilot participants (described in a subsequent section of this document) was equipped with 
NASA’s ASTAR airborne spacing algorithm and the prototype FIM crew interface. 

The ASTAR algorithm produces speed guidance by determining time-to-go until an aircraft and 
its target reach an achieve-by point along a 4D trajectory. During the IM-NOVA experiment, 
ASTAR (version 11.06.22) utilized design features intended to reduce the number of required 
speed changes while maintaining arrival precision. One such feature consisted of a gain schedule 
that made commanded speed changes more sensitive to a given time error as the aircraft 
approached the runway. Another feature consisted of a look-ahead function that looked 10 
seconds ahead for a profile speed decrease and inhibited speed increases during that 10-second 
period. Additionally, ASTAR also included mechanisms intended to keep it from generating 
unacceptable commanded speeds and features intended to help ensure adherence to various 
regulations. For example, to keep the commanded speeds within an acceptable range, ASTAR 
limited commanded speed deviations to ±10 percent of the nominal profile speed and adhered to 
the 250 kt speed restriction below an altitude of 10,000 ft. 

As described previously, the prototype FIM crew interface (shown above in Figure 1 and Figure 
2) consisted of two side-mounted EFBs and two AGDs positioned within the flight crew’s 
forward field of view. The IM application residing on the EFB enabled the flight crew to use the 
EFB’s bezel buttons and touch screen to enter the following information, communicated via an 
ATC radio-issued IM clearance, into the EFB: achieve-by point, STA, ASG, target aircraft 
callsign, and target aircraft route. After entering this information into the EFB, the flight crew 
pressed an “activate” button and then flew speeds to meet their STA until ADS-B information 
from their target aircraft was acquired and FIM operations could begin. 

During FIM operations, the EFB displayed the target aircraft’s callsign; a commanded speed 
(“CMD SPD”), in Mach or knots, to indicate the FIM aircraft speed needed to achieve precise 
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interval spacing behind the target upon reaching the FAF; and a “FAST/SLOW” indication, in 
knots, to provide trend information and guidance regarding required FIM aircraft decelerations 
and accelerations to conform with the ASTAR algorithm. In order to provide the pilot 
participants with key information in an easily viewable location, the AGD supplied the following 
three information elements during an IM operation: 

• A white “IM” light located in the upper left comer of the device that, when illuminated, 
indicated that the FIM aircraft was actively spacing relative to a target aircraft 

• The commanded speed (“CMD SPD”) required to achieve precise interval spacing behind 
a target aircraft 

• A “FAST/SLOW” indication that showed the difference between the FIM aircraft’s 
current speed and the speed that ASTAR expected 

Upon reaching the FAF, the ASTAR spacing algorithm switched to VREF mode, and the flight 
crew began decelerating to their landing speed. 

Appendix E provides an example IM clearance and illustrates the use of the EFB for data entry, 
activation, suspension, resumption, cancellation, and termination of FIM operations. 

Additionally, illustrations of the information elements presented on the AGD during FIM 
operations are provided in Appendix E. 

2.5 Facilities and Equipment 

The IM-NOYA experiment made use of two NASA LaRC facilities: the Air Traffic Operations 
Laboratory (ATOL) and the Integration Flight Deck (IFD). Descriptions of each facility and the 
equipment utilized during the IM-NOVA experiment are provided below. 

2. 5. 1 Air Traffic Operations Laboratory (A TOL) 

The ATOL operates a network of hundreds of real-time, medium-fidelity aircraft simulators and 
utilizes a simulation platform, known as the Airspace and Traffic Operations Simulation 
(ATOS), which can be used for both batch and real-time HITL experiments. During the IM- 
NOVA experiment, the ATOL provided 24 Aircraft Simulation for Traffic Operations Research 
(ASTOR) aircraft, four remote ASTOR pseudo-pilot (RAPP) stations, 25 simulated Multi- 
Aircraft Control System (MACS) aircraft, and four ATC controller stations. 

2.5. 1 . 1 Aircraft Simulation for Traffic Operations Research (ASTOR) 

ASTOR components include: a six degrees-of-freedom aircraft model, primary flight display, 
multi-function display, autopilot and auto-throttle systems, flight management computer, multi- 
function control display unit, mode control panel, and ADS-B (ref. 28). During the IM-NOVA 
experiment, two researchers used a total of four Remote ASTOR Pseudo-Pilot (RAPP) stations 
(described below) to control the 24 ASTOR aircraft that provided multiple air traffic flows into 
DFW. 


2.5.1 .2 Remote ASTOR Pseudo-Pilot (RAPP) stations 

The use of RAPP stations allowed a single operator, or pseudo-pilot, to control the basic 
functions of multiple ASTORs. Each RAPP station displays a GUI (shown in Figure 6) that 
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allows the operator to view a selection list showing each simulated aircraft controlled by that 
RAPP station, a list of ATC frequencies that can be assigned, and the current airspeed, altitude, 
and heading of each aircraft as it is selected. To command a change in airspeed, heading, 
altitude, or frequency, the RAPP operator enters the desired change in the appropriate command 
window and clicks on the corresponding “Send” button. A message containing the desired 
command is then sent to the selected ASTOR. The sent message and a response from the 
ASTOR are displayed in the scratch pad at the bottom of the RAPP station GUI. 

Since the majority of the ASTORs in each scenario independently flew their assigned flight plan 
without ATC intervention, the RAPP station operators primarily handled the radio transmissions 
for their assigned aircraft. However, there were cases in each scenario where the RAPP operators 
executed ATC-directed speed changes to keep aircraft on the TMA arrival schedule. In the ATC 
Termination of FIM scenario, the RAPP station operator executed ATC heading, airspeed, and 
altitude changes required to vector the ASTOR for an approach to runway 13R, although the 
approach and landing were not actually executed. 
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Figure 6. Remote ASTOR Pseudo-Pilot (RAPP) station graphical user interface. 

2.5. 1 .3 Multi-Aircraft Control System (MACS) aircraft 

In addition to the arriving aircraft being simulated with ASTORs, each experiment scenario 
included 25 departing aircraft to provide a more realistic traffic picture for the air traffic 
controllers. The departing aircraft were simulated using the NASA ARC MACS software (ref. 
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29). These aircraft took off from DFW runways 17R and 18L at intervals ranging from 30 
seconds to 120 seconds and were flown solely by a computer-based pilot model. 

2 . 5 . 1 .4 ATC controller stations 

ATC controller stations equipped with MACS were utilized within the ATOL to enable 
confederate air traffic controllers to provide a realistic ATC environment. The confederate 
controllers used either standard Display System Replacement (DSR) or Standard Terminal 
Automation Replacement System (STARS) displays augmented with CMS decision support 
tools. 

A screenshot of the Center controller’s DSR display for an IM-NOVA scenario is shown in 
figure 7. FDBs associated with arrival aircraft are displayed in yellow and include early/late 
indicators. FDBs associated with departure aircraft are displayed in white. 
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Figure 7. IM-NOVA Center controller’s Display System Replacement (DSR) 

display. 

Figure 8 shows a screenshot of the STARS display provided on the IM-NOVA Feeder ATC 
station. White FDBs indicate aircraft owned by the Feeder controller. Green FDBs indicate 
aircraft that are owned by the Final controller, and yellow FDBs are associated with departure 
aircraft. Slot marker circles and speed advisories are displayed for all arrivals, and TMA 
timelines for both runways are displayed on the left and right sides of the STARS display. 


14 






Air Traffic Management Technology Demonstration Phase 1 (ATD-1) Interval Management for Near- 
Term Operations Validation of Acceptability (IM-NOVA) Experiment 



Figure 8. IM-NOVA Feeder controller’s Standard Terminal Automation 

Replacement System (STARS) display. 

2.5.2 Integration Flight Deck (IFD) 

The IFD is a full-scale simulator representative of a large commercial transport category aircraft 
and is driven by an appropriate aircraft dynamics mathematical model (ref. 30). The cockpit 
includes standard instruments representative of a line operations aircraft, and the cockpit’s visual 
system is a panorama system that provides 200° horizontal by 40° vertical field-of-view. During 
the IM-NOVA experiment, all pilot participants flew the IFD, and the visual scene used was the 
DFW terminal environment in a daytime setting. As noted previously, the IFD simulator was 
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equipped with the ASTAR algorithm and the prototype FIM crew interface to enable flight crews 
to perform FIM operations. 

Figure 9 shows the positions of the EFBs and AGDs within the IFD. 



Figure 9. Integration Flight Deck (IFD) equipped with IM-NOVA’s prototype FIM 

crew interface. 

2.6 Dependent Measures 

To assess the acceptability of the procedures and test Hypotheses 1-7, qualitative data were 
collected from pilots via electronic questionnaires. Following each run, the pilots were asked to 
rate the level of workload during the scenario they had just completed using the MCH Subjective 
Workload Rating Scale. They were also asked to characterize the following: the acceptability of 
the use of the procedures in a voice communications environment, the completeness of the 
procedures, the usability of the crew interface, the acceptability of the data entry procedures, and 
the operational acceptability of the rate of speed commands. At the end of the second day of the 
two-day experiment session, each flight crew participated in an interactive group debrief session 
with the research team. 

To assess the feasibility of the procedures and test Hypotheses 7 and 8, quantitative data, 
including a spacing error at the FAF and the rate of ASTAR commanded speed changes, were 
collected during each run. Although the achieve-by point was the FAF, the spacing error at the 
runway threshold was an additional metric of interest. Since the ATD-1 procedures used in this 
experiment required that a complex voice clearance be issued to the flight crew by ATC using 
radio communications, data were collected during each run to allow an assessment of the time 
required for clearance issuance and read back, as well as the time required for the flight crew to 
enter information included in the clearance into the EFB. 
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3 Results and Discussion 

To assess the acceptability of the proposed ATD-1 flight crew procedures, subjective response 
data were collected in the form of acceptability, workload, and usability ratings via electronic 
post-run and post-experiment questionnaires (questionnaires are included in Appendix C and 
Appendix D). To assess the feasibility of the procedures, quantitative data were collected, 
including the rate of speed commands and the spacing error at the FAF. An a priori hypothesis 
associated with the spacing error at the runway threshold was not generated; however, a spacing 
error at the runway threshold was computed for each run. Similarly, a priori hypotheses 
associated with data collected during the exploratory scenario were not generated; however, 
descriptive statistics calculated using the exploratory scenario data are included in Table 2-Table 
13. 

Results from pilot participant crew #2, Scenario 5 (ADS-B Loss), with the first officer as PF and 
the captain as PM, were excluded from this analysis due to a simulation error. 2 Results 
associated with the time required for IM clearance issuance and read back as well as the time 
required for the flight crew to enter information included in the clearance into the EFB are 
reported elsewhere (ref. 31). 

3.1 Acceptability of the Procedures 

Hypothesis 1 is as follows: the use of the procedures for receiving and executing IM 
clearances in a voice communications environment will be acceptable to the flight crew, 
with mean ratings of the procedures’ completeness and acceptability higher than 4.5 on a 
7-point scale. 

To assess Hypothesis 1, data from the following two items of the post-run questionnaires were 
used: 

5f. The use of voice communications to provide the IM clearance(s) was acceptable 
in this scenario. 

5j. The flight crew procedures for the events in this scenario were complete and 
acceptable. 

Descriptive statistics associated with the pilot participants’ acceptability ratings are shown in 
Table 2 and Table 3. Statistical analysis was performed using the Wilcoxon signed rank test, a 
nonparametric test appropriate for analyzing ordinal data (ref. 31). There were no statistically 
significant differences between the mean responses from the PF and PM for either questionnaire 
item 5f or 5j in any scenario (p > 0.236), 3 except for questionnaire item 5j in the Nominal 
scenario (p = 0.036). In this case, the mean acceptability rating was slightly higher for the PM 
(mean = 6.9, SD = 0.3) than for the PF (mean = 6.5, SD = 0.9). For all five scenarios, both the PF 
and PM generally found the proposed ATD-1 procedures to be acceptable in a voice 
communications environment (p < 0.002). However, concerns regarding the acceptability of the 


2 Researchers responsible for communicating information regarding the target aircraft’s loss of ADS-B capability to 
pilot participant crew #2 failed to provide this information at the appropriate time during the experiment scenario. 

3 A p- value < 0.05 indicates a statistically significant difference between sample means. 
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proposed procedures were raised by several pilot participants as evidenced by a total of 10 
ratings of 3 (Slightly Disagrees) or less recorded in response to questionnaire item 5f and a total 
of two ratings of 3 or less recorded in response to questionnaire item 5j. These low ratings of 
acceptability were consistently associated with pilots’ reported uneasiness with the issuance of 
IM clearances below 10,000 ft. Specific pilot participant comments referenced concerns with 
“task saturation below 10,000 ft” and a hesitancy to enter data associated with an IM clearance 
into an EFB while completing other tasks required within a potentially “high demand terminal 
environment.” 


Table 2. Descriptive statistics for acceptability ratings (from post-run 

questionnaire item 5f). 


Scenario 

N 

PF 

PM 

Mean 

SD 

Min 

Med 

Max 

Mean 

SD 

Min 

Med 

Max 

Nominal 

20 

6.7 

0.7 

4 

7 

7 

6.6 

0.9 

3 

7 

7 

Amend ASG 

20 

6.4 

1.1 

3 

7 

7 

6.4 

1.2 

2 

7 

7 

ATC Terminates 

20 

6.8 

0.9 

3 

7 

7 

6.6 

1.0 

3 

7 

7 

Suspend/Resume 

20 

6.9 

0.4 

6 

7 

7 

6.7 

0.7 

5 

7 

7 

ADS-B Loss 

19 

6.2 

1.5 

2 

7 

7 

6.1 

1.6 

2 

7 

7 

Exploratory 

10 

6.9 

0.3 

6 

7 

7 

6.6 

1.0 

4 

7 

7 


Table 3. Descriptive statistics for acceptability ratings (from post-run 

questionnaire item 5j). 


Scenario 

N 

PF 

PM 

Mean 

SD 

Min 

Med 

Max 

Mean 

SD 

Min 

Med 

Max 

Nominal 

20 

6.5 

0.9 

4 

7 

7 

6.9 

0.3 

6 

7 

7 

Amend ASG 

20 

6.6 

0.7 

5 

7 

7 

6.6 

0.8 

4 

7 

7 

ATC Terminates 

20 

6.7 

0.6 

5 

7 

7 

6.6 

0.9 

3 

7 

7 

Suspend/Resume 

20 

6.5 

0.8 

5 

7 

7 

6.8 

0.5 

5 

7 

7 

ADS-B Loss 

19 

6.5 

1.0 

4 

7 

7 

6.3 

1.4 

2 

7 

7 

Exploratory 

10 

6.9 

0.3 

6 

7 

7 

6.8 

0.4 

6 

7 

7 


3.2 Completeness of the Procedures 

Hypothesis 2 is as follows: pilots will report that the procedures are complete (i.e., at 
least 90 percent of the pilots will report that the procedures are complete). 
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To assess Hypothesis 2, data from the following three items of the post-run questionnaires were 
used: 

6a. The procedures did not contain missing steps. 

6b. The procedures did not contain extra steps that were unnecessary. 

6c. The procedural steps were logical and easy to follow. 

Table 4 shows the proportion of pilots who agreed with questionnaire items 6a, 6b, and 6c. Using 
Fisher’s exact test, there were no statistically significant differences between the proportion of 
PF and PM who reported the procedures were complete in any scenario (p > 0.487) (ref. 31). 
Statistical analysis was performed using the binomial test of one proportion to test whether the 
proportion of pilots who reported the procedures were complete was at least 90 percent vs. less 
than 90 percent (ref. 31). For all five scenarios, the proportion of pilot participants who reported 
that the procedures contained no missing steps, no unnecessary steps, and that the steps were 
logical and easy to follow was not significantly less than 90 percent (p > 0.580). This implies 
that the pilots felt the procedures were complete. 


Table 4. Percentage (%) of pilots who reported procedures were complete (from 

post-run questionnaire items 6a, 6b, and 6c). 


Scenario 

N 

Questionnaire 
Item 6a 

Questionnaire 
Item 6b 

Questionnaire 
Item 6c 

PF 

PM 

PF 

PM 

PF 

PM 

Nominal 

20 

95.0 

100.0 

100.0 

90.0 

95.0 

100.0 

Amend ASG 

20 

95.0 

95.0 

95.0 

90.0 

100.0 

100.0 

ATC Terminates 

20 

100.0 

95.0 

100.0 

100.0 

100.0 

90.0 

Suspend/Resume 

20 

95.0 

100.0 

100.0 

100.0 

100.0 

100.0 

ADS-B Loss 

19 

89.5 

94.7 

100.0 

94.7 

100.0 

94.7 

Exploratory 

10 

100.0 

100.0 

100.0 

90.0 

100.0 

100.0 


3.3 Pilot Workload 

Hypothesis 3 is as follows: pilots will report the workload level required to execute the 
procedures to be acceptable, with a mean rating of less than 3 on the MCH rating scale. 

Hypothesis 4 is as follows: pilots will report no increase in workload with the Amend 
ASG, ATC Termination of FIM, Suspend/Resume FIM, or ADS-B Loss scenarios as 
compared to the Nominal scenario (i.e., a difference of less than 1 unit on the MCH 
rating scale). 

To assess Hypotheses 3 and 4, workload rating data collected using the MCH flow chart 
presented in the post-run questionnaires were used. 
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Descriptive statistics associated with the pilot participants’ workload ratings are shown in Table 
5. Statistical analysis was performed using the Wilcoxon signed rank test. There were no 
statistically significant differences between the mean responses from the PF and PM in any 
scenario (p > 0.221). For all five scenarios, both the PF and PM found the workload level 
required to execute the procedures to be acceptable (p < 0.0005). Pilots also reported no 
significant increase in the workload for the Amend ASG, ATC Termination of FIM, 
Suspend/Resume FIM, or ADS-B Loss scenarios as compared to the Nominal scenario for either 
PF or PM (p < 0.003). 


Table 5. Descriptive statistics for pilot workload ratings (from post-run 

questionnaire item 3). 


Scenario 

N 

PF 

PM 

Mean 

SD 

Min 

Med 

Max 

Mean 

SD 

Min 

Med 

Max 

Nominal 

20 

1.6 

0.7 

1 

1 

3 

1.5 

0.6 

1 

1 

3 

Amend ASG 

20 

1.6 

0.6 

1 

2 

3 

1.3 

0.5 

1 

1 

2 

ATC Terminates 

20 

1.5 

0.5 

1 

1 

2 

1.6 

0.6 

1 

2 

3 

Suspend/Resume 

20 

1.5 

0.7 

1 

1 

3 

1.5 

0.6 

1 

1 

3 

ADS-B Loss 

19 

1.7 

0.7 

1 

2 

3 

1.8 

0.8 

1 

2 

3 

Exploratory 

10 

1.3 

0.5 

1 

1 

2 

1.3 

0.7 

1 

1 

3 


3.4 Usability of the Crew Interface 

Hypothesis 5 is as follows: pilots will report that the crew interface was usable, with 
mean ratings of the acceptability of heads-down time required and usability of the 
displays will be higher than 4.5 on a 7-point scale. 

To assess Hypothesis 5, data from the following two items of the post-run questionnaires were 
used: 

5g. The amount of heads-down time required to input information from the IM clearance(s) 
into the EFB was acceptable. 

5i. During this scenario, it was easy to obtain needed information from the IM displays. 

Descriptive statistics associated with the pilot participants’ ratings of the amount of heads-down 
time and display usability are shown in Table 6 and Table 7. Statistical analysis was performed 
using the Wilcoxon signed rank test. There were no statistically significant differences between 
the mean responses from the PF and PM for either questionnaire item 5g or 5i in any scenario (p 
> 0.141). For all five scenarios, pilots reported the amount of heads-down time required to input 
information from the IM clearance(s) into the EFB was acceptable (p < 0.001), and found that it 
was easy to obtain needed information from the IM displays (p < 0.0005). This indicates that the 
pilots generally felt that the prototype FIM crew interface was usable. However, several pilots 
expressed concerns regarding the usability of the crew interface as evidenced by a total of six 
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ratings of 3 or less recorded in response to questionnaire item 5g and a total of four ratings of 3 
in response to questionnaire item 5i. Specific concerns focused on the amount of heads-down 
time required to input information from an IM clearance into the EFB during operations 
occurring below 10,000 ft and the limited amount of pertinent information presented within the 
forward field of view by the AGD. 


Table 6. Descriptive statistics for ratings of the amount of heads-down time 

(from post-run questionnaire item 5g). 


Scenario 

N 

PF 

PM 

Mean 

SD 

Min 

Med 

Max 

Mean 

SD 

Min 

Med 

Max 

Nominal 

20 

6.5 

0.7 

5 

7 

7 

6.8 

0.4 

6 

7 

7 

Amend ASG 

20 

6.6 

1.0 

3 

7 

7 

6.6 

0.7 

5 

7 

7 

ATC Terminates 

20 

6.6 

0.8 

4 

7 

7 

6.3 

1.4 

1 

7 

7 

Suspend/Resume 

20 

6.6 

0.8 

4 

7 

7 

6.8 

0.6 

5 

7 

7 

ADS-B Loss 

19 

6.0 

1.4 

3 

7 

7 

6.1 

1.6 

2 

7 

7 

Exploratory 

10 

6.4 

1.0 

4 

7 

7 

6.6 

0.7 

5 

7 

7 


Table 7. Descriptive statistics for ratings of the usability of the displays (from 

post-run questionnaire item 5i). 


Scenario 

N 

PF 

PM 

Mean 

SD 

Min 

Med 

Max 

Mean 

SD 

Min 

Med 

Max 

Nominal 

20 

6.5 

1.0 

3 

7 

7 

6.6 

0.7 

5 

7 

7 

Amend ASG 

20 

6.7 

0.5 

6 

7 

7 

6.4 

1.1 

3 

7 

7 

ATC Terminates 

20 

6.6 

0.7 

5 

7 

7 

6.7 

0.7 

5 

7 

7 

Suspend/Resume 

20 

6.7 

0.6 

5 

7 

7 

6.7 

0.6 

5 

7 

7 

ADS-B Loss 

19 

6.3 

1.1 

3 

7 

7 

6.5 

1.1 

3 

7 

7 

Exploratory 

10 

6.9 

0.3 

6 

7 

7 

6.8 

0.4 

6 

7 

7 


3.5 Acceptability of Data Entry Procedures 

Hypothesis 6 is as follows: the data entry procedures will be acceptable to the flight crew, 
with mean ratings of ease and intuitiveness of entering information being higher than 4.5 
on a 7-point scale. 
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To assess Hypothesis 6, data from the following post-run questionnaire item were used: 

5h. During this scenario, entering IM clearance information into the EFB was easy 
and intuitive. 

Descriptive statistics associated with the pilot participants’ ratings of the ease and intuitiveness 
of entering information are shown in Table 8. Statistical analysis performed using the Wilcoxon 
signed rank test found no statistically significant differences between the mean responses from 
the PF and PM in any scenario ip > 0.100). For all five scenarios, pilots found the data entry 
procedures to be acceptable ip < 0.0005). However, three ratings of 3 recorded in response to 
questionnaire item 5h were associated with concerns expressed regarding the input of data into 
the EFB at low altitudes within the terminal environment. 


Table 8.Descriptive statistics for ratings of the ease and intuitiveness of entering 

information (from post-run questionnaire item 5h). 


Scenario 

N 

PF 

PM 

Mean 

SD 

Min 

Med 

Max 

Mean 

SD 

Min 

Med 

Max 

Nominal 

20 

6.6 

0.8 

4 

7 

7 

6.9 

0.3 

6 

7 

7 

Amend ASG 

20 

6.4 

1.1 

4 

7 

7 

6.5 

1.0 

4 

7 

7 

ATC Terminates 

20 

6.6 

0.9 

4 

7 

7 

6.6 

1.0 

3 

7 

7 

Suspend/Resume 

20 

6.7 

0.7 

4 

7 

7 

6.8 

0.4 

6 

7 

7 

ADS-B Foss 

19 

6.3 

1.2 

4 

7 

7 

6.3 

1.3 

3 

7 

7 

Exploratory 

10 

6.6 

1.0 

4 

7 

7 

6.8 

0.6 

5 

7 

7 


3.6 Rate of Speed Commands 

Hypothesis 7 is as follows: the rate of speed commands will be acceptable to the flight 
crew, with speed commands occurring at a rate of less than two per minute, and mean 
ratings of operational acceptability and acceptability of the frequency of speed commands 
being higher than 4.5 on a 7-point scale. 

To assess Hypothesis 7, the rate of speed commands during each segment of flight as well as 
data from the following two post-run questionnaire items were used: 

5c. The IM commanded speeds were operationally acceptable and appropriate. 

5d. The frequency of the IM speed commands was acceptable at all times throughout the 
scenario. 

For each run, the rate of speed commands was computed. Table 9 presents the mean rate of 
speed commands over the entire flight, as well as for each segment of flight: from Flight Fevel 
(FL) 240 to FL180, FL180 to 1 1,000 ft, 1 1,000 ft to 6,000 ft, and 6,000 ft to the FAF. Statistical 
analysis was performed using the one-sample Poisson rate test, a hypothesis test appropriate for 
analyzing the number of occurrences of an event in a given length of time (ref. 33). For all five 
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scenarios, the mean rate of speed commands was less than two per minute for each flight 
segment (p < 0.0005). 
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Table 10 and 

Table 11 show descriptive statistics associated with the pilot participants’ ratings for 
questionnaire items 5c and 5d. Statistical analysis was performed using the Wilcoxon signed 
rank test. There were no statistically significant differences between the mean responses from the 
PF and PM for either questionnaire item 5c or 5d in any scenario (p > 0.205). For all five 
scenarios, pilots generally reported that the IM commanded speeds were operationally acceptable 
and appropriate (p < 0.001), and pilots found the frequency of the IM speed commands to be 
acceptable (p < 0.001). When referencing Table 9 and 
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Table 10, note that only two of the 20 pilots provided ratings of 3 or less during the experiment 
scenarios. One pilot commented that the issuance of a new clearance below 10,000 ft during the 
ADS-B Loss scenario resulted in too much heads-down time by the PM, and comments from the 
other pilot indicated that two speed commands occurred in less than five seconds during two of 
the scenarios. 


Table 9. Mean rate of speed commands (number of speed commands per 

minute) for each segment of flight. 


Scenario 

N 

FL240 to 180 

FL180 to 
11,000 ft 

11,000 ft to 
6,000 ft 

6,000 ft to 
FAF 

Total 

Nominal 

20 

0.22 

0.44 

0.35 

0.76 

0.49 

Amend ASG 

20 

0.09 

0.65 

0.53 

0.67 

0.54 

ATC Terminates 

20 

0.12 

0.50 

n/a 

0.67 

0.53 

Suspend/Resume 

20 

0.13 

0.40 

0.35 

0.61 

0.43 

ADS-B Loss 

19 

n/a* 

0.43 

n/a 

0.72 

0.61 

Exploratory 

10 

0.29 

0.60 

0.63 

0.70 

0.59 


*Note that mean rate = “n/a” indicates aircraft was conducting FIM operations less than 75% of the time. 
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Table 10. Descriptive statistics for ratings of operational acceptability and 
appropriateness of IM commanded speeds (from post-run 
questionnaire item 5c). 


Scenario 

N 

PF 

PM 

Mean 

SD 

Min 

Med 

Max 

Mean 

SD 

Min 

Med 

Max 

Nominal 

20 

6.3 

1.3 

2 

7 

7 

6.8 

0.4 

6 

7 

7 

Amend ASG 

20 

6.9 

0.4 

6 

7 

7 

6.7 

0.6 

5 

7 

7 

ATC Terminates 

20 

6.6 

0.8 

4 

7 

7 

6.5 

1.4 

1 

7 

7 

Suspend/Resume 

20 

6.5 

0.9 

4 

7 

7 

6.8 

0.4 

6 

7 

7 

ADS-B Loss 

19 

6.3 

1.5 

1 

7 

7 

6.4 

1.3 

2 

7 

7 

Exploratory 

10 

6.7 

0.7 

5 

7 

7 

6.5 

0.7 

5 

7 

7 


Table 1 1 . Descriptive statistics for ratings of acceptability of the frequency of 
the IM speed commands (from questionnaire item 5d). 


Scenario 

N 

PF 

PM 

Mean 

SD 

Min 

Med 

Max 

Mean 

SD 

Min 

Med 

Max 

Nominal 

20 

6.6 

0.6 

5 

7 

7 

6.7 

0.5 

6 

7 

7 

Amend ASG 

20 

6.6 

0.8 

5 

7 

7 

6.5 

0.9 

4 

7 

7 

ATC Terminates 

20 

6.5 

0.8 

4 

7 

7 

6.2 

1.4 

1 

7 

7 

Suspend/Resume 

20 

6.6 

0.8 

4 

7 

7 

6.6 

0.8 

4 

7 

7 

ADS-B Loss 

19 

6.4 

1.5 

1 

7 

7 

6.5 

1.1 

3 

7 

7 

Exploratory 

10 

6.6 

0.7 

5 

7 

7 

6.3 

1.3 

3 

7 

7 


3.7 Spacing Error at the Final Approach Fix 

For each run, the spacing error at the FAF was recorded to assess the following a priori 
hypothesis: 

Hypothesis 8: The spacing error at the FAF will be within ±5 sec with a standard 
deviation of 5 sec. 

Descriptive statistics associated with the spacing error at the FAF are provided in Table 12, and 
histograms for each scenario are shown in Figure 150-Figure 19. The one-sample /-test and one- 
sample variance test were used to test the mean and standard deviation, respectively (ref. 33). For 
all five scenarios, the spacing error at the FAF had a mean within +5 sec (p < 0.0005) and a 
standard deviation less than 5 sec (p < 0.0005). 
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Table 12. Descriptive statistics for spacing error at the FAF (sec). 


Scenario 

N 

Mean 

SD 

Min 

Median 

Max 

Nominal 

20 

-1.7 

1.0 

-4.1 

-1.5 

-0.1 

Amend ASG 

20 

-1.8 

1.3 

-5.3 

-1.6 

0.5 

ATC Terminates 

20 

-0.8 

1.1 

-3.4 

-0.9 

0.9 

Suspend/Resume 

20 

1.2 

1.5 

-1.7 

1.5 

3.4 

ADS-B Loss 

19 

-0.5 

0.9 

-1.6 

- 0.7 

1.7 

Exploratory 

10 

-1.3 

1.4 

-3.0 

-1.4 

1.5 


Nominal 



Spacing Error at the FAF (sec) 


Figure 10. Spacing error at the FAF 
(sec) for the Nominal 
scenario. 


ATC Terminates 



Spacing Error at the FAF (sec) 


Figure 12. Spacing error at the FAF 
(sec) for the ATC 
Termination of FIM 
scenario. 



Spacing Error at the FAF (sec) 

Figure 11. Spacing error at the FAF 
(sec) for the Amend ASG 
scenario. 



Spacing Error at the FAF (sec) 

Figure 13. Spacing error at the FAF 
(sec) for the 
Suspend/Resume 
scenario. 
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ADS-B Loss 



Figure 14. Spacing error at the FAF 
(sec) for the ADS-B Loss 
scenario. 

3.8 Spacing Error at the Runway Threshold 

The spacing error at the runway threshold was computed for each run. Table 13 gives descriptive 
statistics associated with this metric, and Figure 15-Figure 19 show histograms for each 
scenario. For all five scenarios, the observed mean spacing error at the runway threshold was 
within ±2 sec, and the observed standard deviation was less than 3 sec. 


Table 13. Descriptive statistics for spacing error at the runway threshold (sec). 


Scenario 

N 

Mean 

SD 

Min 

Med 

Max 

Nominal 

20 

-0.7 

2.2 

-4.1 

-0.6 

3.8 

Amend ASG 

20 

-0.5 

2.9 

-8.5 

-0.2 

5.0 

ATC Terminates 

20 

-1.6 

2.3 

-6.3 

-1.7 

1.9 

Suspend/Resume 

20 

0.4 

2.9 

-5.8 

0.8 

5.3 

ADS-B Loss 

19 

-1.3 

2.5 

-5.1 

-1.8 

5.1 

Exploratory 

10 

-0.7 

2.0 

-4.8 

0.2 

0.9 
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Spacing Error at the Runway Threshold (sec) 

Figure 15. Spacing error at the 

runway threshold (sec) for 
the Nominal scenario. 


ATC Terminates 



Spacing Error at the Runway Threshold (sec) 


Figure 17. Spacing error at the 

runway threshold (sec) for 
the ATC Termination of 
FIM scenario. 


ADS-B Loss 



Spacing Error at the Runway Threshold (sec) 

Figure 19. Spacing error at the 

runway threshold (sec) for 
the ADS-B Loss scenario. 



-8 -6 -4 -2 0 2 4 

Spacing Error at the Runway Threshold (sec) 


Figure 16. Spacing error at the 

runway threshold (sec) for 
the Amend ASG scenario. 


Suspend/Resume 

30 - 



Spacing Error at the Runway Threshold (sec) 

Figure 18. Spacing error at the 

runway threshold (sec) for 
the Suspend/Resume 
scenario. 
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4 Conclusions 

NASA has developed a set of ground-based and airborne arrival management technologies, 
including TMA-TM, CMS decision support tools, and FIM avionics and procedures. The 
integration of these technologies will increase throughput, reduce delay, and minimize 
environmental impacts. ATD-1 will operationally demonstrate the efficient arrival operations 
provided by this integrated system of NextGen technologies. 

The HITL experiment described in Section 1.1 was conducted as part of initial preparations for 
an ATD-1 flight demonstration. It was designed to assess the acceptability and feasibility of the 
proposed air/ground procedures in a voice communications environment. Five flight scenarios 
were defined to allow flight crews to fully exercise the procedures during different flight phases 
and operational events. These scenarios consisted of a nominal IM clearance flown to landing, an 
amended IM clearance on arrival, a terminated IM clearance with a reissue of a new clearance, a 
suspension and resumption of the IM clearance, and a system error causing a flight crew 
termination of IM (ADS-B loss) with a subsequent new clearance issued at low altitude (below 
10,000 ft). 

Overall, the FIM crew procedures were deemed acceptable and feasible for use by the flight crew 
in all scenarios and phases of flight flown in the experiment. Analyses of qualitative data 
revealed that the procedures used by flight crews to receive and execute IM clearances in a voice 
communications environment were logical, easy to follow, did not contain any missing or 
extraneous steps, and required the use of an acceptable level of workload. The majority of the 
pilot participants found the IM concept and the proposed FIM crew procedures to be acceptable 
and indicated that the ATD-1 procedures could be successfully executed in a near-term NextGen 
environment. Analyses of quantitative data revealed that FIM speed commands occurred at a rate 
of less than two per minute, and pilot participants found the frequency of the speed commands to 
be acceptable at all times throughout the experiment scenarios. Pilots also reported that the IM 
commanded speeds were operationally acceptable and appropriate during all scenarios. In 
addition, the delivery accuracy at both the FAF and the runway threshold was within ±5 sec, and 
the delivery precision was less than 5 sec. The results of this experiment demonstrate the 
effectiveness of the airborne spacing algorithm and the air/ground procedures investigated. 

Future research is planned to investigate the effects of winds, weather, and turbulence on the 
acceptability and feasibility of the ATD-1 air/ground procedures. 
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Appendix A. Simulated Airspace and Scenario Design 

The Interval Management for Near-Term Operations Validation of Acceptability (IM-NOVA) 
experiment’s common scenario consisted of 25 arrival aircraft created at various points on the 
arrival routes into Dallas-Fort Worth (DFW) airport and landing on runways 17C and 18R. To 
provide a realistic traffic environment, each scenario also included 25 departure aircraft 
generated by the Multi- Aircraft Control System (MACS) developed at NASA Ames Research 
Center (ARC). 

All IM-NOVA inbound aircraft approached the DFW airport via one of four standard terminal 
arrival routes (STARs) with a total of 10 different transitions. The distribution of aircraft over the 
STARs/Transitions is shown in Table A-l, and the research STAR charts are included in Figure 
A- 1 -Figure A-4. 


Table A-1 . Distribution of aircraft over the STARs/Transitions. 


STAR 

Transition 

Number of Aircraft 

Bonham Five 

Fort Smith 

3 


Little Rock 

2 


Paris 

1 

Bowie One 

Borger 

5 


Texico 

2 

Cedar Creek Six 

Alexandria 

2 


Gregg County 

2 


Humble 

2 

Glenn Rose Nine 

San Antonio 

5 


Wink 

1 
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Figure A-1. NASA Research STAR for the Bonham Five Arrival into DFW. 
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Figure A-2. NASA Research STAR for the Bowie One Arrival into DFW. 
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Figure A-3. NASA Research STAR for the Cedar Creek Six Arrival into DFW. 
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Figure A-4. NASA Research STAR for the Glen Rose Nine Arrival into DFW. 

Each inbound aircraft was represented by an instantiation of the NASA Langley Research Center 
(LaRC) Aircraft Simulation for Traffic Operations Research (ASTOR). Each ASTOR 
instantiation represents a commercial transport aircraft, its flight deck systems, and the airborne 
components of a realistic future communications, navigation, and surveillance infrastructure and 
runs on a single Windows XP workstation. Some aircraft initialized in level cruise and flew the 
full arrival and approach to the runway, while others initialized in descent and flew only a 
portion of the arrival before flying the approach. All aircraft started outside the DFW Terminal 
Radar Approach Control (TRACON) boundary so that the Traffic Management Advisor with 
Terminal Metering (TMA-TM) could develop a workable arrival schedule. 

Five experiment scenarios were developed from a common scenario in order to assess the 
viability of the IM spacing procedures during each of the operational scenarios defined in the 
ATD-1 Concept of Operations (ConOps). Each experiment scenario included the same 25 aircraft 
defined in the common scenario, with one ASTOR in each scenario replaced by the NASA LaRC 
Integration Flight Deck (IFD): a full-scale simulator representative of a large commercial 
transport category aircraft flown by the pilot participants. The pilot participants’ (i.e., Flight 
Deck-based Interval Management (FIM)) aircraft were changed for each scenario to provide 
variety, exercise all use cases defined in the ATD-1 ConOps, and keep the scenario length to less 
than 30 minutes. The closest aircraft in the arrival stream for the same runway as the FIM aircraft 
was designated as the initial IM spacing target. If a new target was designated later in the 
scenario, it was always the next closest aircraft in the arrival stream. 
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The experiment scenarios were as follows: 

1. Nominal Scenario : ATC issued an IM clearance before top of descent (TOD). After the 
FIM aircraft reported that spacing had commenced, the FIM aircraft maintained nominal 
IM spacing operations until passing the final approach fix (FAF). 



Figure A-5. Nominal Scenario. 
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2. Amend Assigned Spacing Goal (ASG) : The initial IM clearance was issued shortly after 
TOD. Approximately two minutes after the FIM aircraft reported that spacing had 
commenced, ATC issued an amended clearance to increase spacing by 20 seconds. 



Figure A-6. Amend ASG Scenario. 
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3. ATC Termination of FIM : The initial IM clearance was issued shortly after TOD. After 
both the target and FIM aircraft were well inside the DFW TRACON, ATC cancelled the 
IM clearance and vectored the target aircraft for landing on runway 13R. ATC then 
issued a new IM clearance with a new target for the FIM aircraft. 



Figure A-7. ATC Termination of FIM Scenario. 
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4. Suspend / Resume FIM : The initial IM clearance was issued shortly after TOD. After the 
FIM aircraft reported that spacing had commenced, ATC suspended the IM clearance and 
issued a speed change of 20 kt for the FIM aircraft. Approximately two minutes later, 
ATC cleared the FIM aircraft to resume IM spacing. 



Figure A-8. Suspend / Resume FIM Scenario. 
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5. ADS-B Loss : ATC issued the initial IM clearance before TOD. After the FIM aircraft 
reported that spacing had commenced and both aircraft were well within the DFW 
TRACON, the target aircraft experienced a loss of ADS-B capability. After the pilot 
participants notified ATC that they were “IM Unable” due to the loss of the target 
aircraft’s ADS-B signal, ATC cancelled the initial IM clearance and issued a new 
clearance with a new target for the FIM aircraft. 



Figure A-9. ADS-B Loss Scenario. 


In addition to the five experiment scenarios, three training scenarios were developed to allow the 
pilot participants to practice the IM spacing procedures prior to beginning the experiment’s data 
collection runs. These training scenarios contained all of the elements of the experiment 
scenarios but with a different sequence of events and different initial conditions. The exploratory 
scenario that pilot participants were asked to fly after completing all experiment scenarios 
consisted of the Amend ASG scenario with the addition of aural alerts associated with 
commanded speed changes from the Airborne Spacing for Terminal Area Routes (ASTAR) 
algorithm. 
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Appendix B. Experiment Run Order 

Each two-person crew (with both members employed by the same airline) participated in a two- 
day experiment session. Every crew flew each scenario twice: once with the captain as the pilot 
flying (PF) and the first officer as the pilot monitoring (PM), and once with the first officer as the 
PF and the captain as the PM. One additional exploratory scenario was created to examine the 
effect of aural alerts associated with Airborne Spacing for Terminal Area Routes (ASTAR) 
algorithm commanded speed changes. Therefore, each crew flew a total of 1 1 runs. 

This experiment utilized two 2-replicate Latin square designs 4 with “crew” and “run order” as 
the two blocking factors (see Table B-l). The first 2-replicate Latin square design was used to 
partially counterbalance the run order of the scenarios with the captain as PF and the first officer 
as PM. The second 2-replicate Latin square design was used for the scenarios with the first 
officer as PF and the captain as PM. These two designs are shown below, where “crew” 
represents the row blocks and “run number” represents the column blocks. Within each crew, the 
pilots switched responsibilities between runs. For example, crew 1 flew scenario 5 with the 
captain as PF, then scenario 4 with the first officer as PF, followed by scenario 2 with the captain 
as PF, etc. This resulted in the run order shown in Table B-2. The exploratory scenario was 
always flown after the pilot participants had completed all 10 data collection runs. 

Table B-1. Latin square designs. 


Captain as PF and First Officer as PM 
Run 

1 Run3 Run5 Run7 Run9 

Crew 1 

5 

2 

1 

4 

3 

Crew 2 

3 

4 

5 

2 

1 

Crew 3 

1 

3 

2 

5 

4 

Crew 4 

2 

1 

4 

3 

5 

Crew 5 

4 

5 

3 

1 

2 

Crew 6 

2 

1 

3 

5 

4 

Crew 7 

3 

4 

2 

1 

5 

Crew 8 

4 

3 

5 

2 

1 

Crew 9 

5 

2 

1 

4 

3 

Crew 10 

1 

5 

4 

3 

2 


First Officer as PF and Captain as PM 

Run 

Run2 Run4 Run6 Run8 10 

Crew 1 

4 

3 

2 

1 

5 

Crew 2 

3 

2 

5 

4 

1 

Crew 3 

2 

4 

1 

5 

3 

Crew 4 

1 

5 

4 

3 

2 

Crew 5 

5 

1 

3 

2 

4 

Crew 6 

1 

4 

5 

3 

2 

Crew 7 

2 

5 

1 

4 

3 

Crew 8 

3 

1 

2 

5 

4 

Crew 9 

5 

3 

4 

2 

1 

Crew 10 

4 

2 

3 

1 

5 


4 Dean, A. and Voss, D.. Design and Analysis of Experiments. Springer-Verlag New York, Inc., New York, NY, 
1999 . 
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Table B-2. Experiment run order. 


PF: 

Runl 

Capt 

Run2 

F/O 

Run3 

Capt 

Run4 

F/O 

Run5 

Capt 

Run6 

F/O 

Run7 

Capt 

Run8 

F/O 

Run9 

Capt 

Run 10 
F/O 

Crew 1 

5 

4 

2 

3 

1 

2 

4 

1 

3 

5 

Crew 2 

3 

3 

4 

2 

5 

5 

2 

4 

1 

1 

Crew 3 

1 

2 

3 

4 

2 

1 

5 

5 

4 

3 

Crew 4 

2 

1 

1 

5 

4 

4 

3 

3 

5 

2 

Crew 5 

4 

5 

5 

1 

3 

3 

1 

2 

2 

4 

Crew 6 

2 

1 

1 

4 

3 

5 

5 

3 

4 

2 

Crew 7 

3 

2 

4 

5 

2 

1 

1 

4 

5 

3 

Crew 8 

4 

3 

3 

1 

5 

2 

2 

5 

1 

4 

Crew 9 

5 

5 

2 

3 

1 

4 

4 

2 

3 

1 

Crew 10 

1 

4 

5 

2 

4 

3 

3 

1 

2 

5 
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Appendix C. Post-Run Questionnaire 

Pilot participants were asked to complete the following post-run questionnaire immediately 
following each of the experiment’s scenarios: 


This questionnaire is intended to capture measures of workload and ratings for the events that 
occurred in the scenario that was just completed. You will be asked to complete a more extensive 
questionnaire at the end of the experiment, so please try and keep written comments as concise 
as possible. 

1. Please circle the scenario you just completed from the list below: 

• Scenario 1 

• Scenario 2 

• Scenario 3 

• Scenario 4 

• Scenario 5 

• Scenario 6 

• Scenario 7 

• Scenario 8 

• Scenario 9 

• Scenario 10 

2. Please circle your role during the scenario you just completed from the list below: 

• Pilot Flying 

• Pilot Not Flying / Pilot Monitoring 
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Average Workload Ratings (Modified Cooper-Harper) 


Difficulty Level 


Operator Demand Level 


Operator Mental Effort Is Minimal And 
Desired Performance Is Easily Attainable 



3. Follow the flow chart above to select the average workload you experienced during the 
scenario you just completed. 

> Rating of your average workload level: 

(Optional) Use the space provided below to record any clarifying comments or interesting 
observations related to the workload level you experienced during the last run: 
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Peak Workload Ratings (Modified Cooper-Harper) 

Difficulty Level Operator Demand Level Rating 



Even Though 
Errors May Be 
Large Or Frequent, 
Can Instructed Task 
BeAccomplished 
Most Of The Time? 



Very Easy, 
Highly Desirable 

Operator Mental Effort Is Minimal And 
Desired Performance Is Easily Attainable 

Easy, 

Operator Mental Effort Is Low 

Desirable 

And Desired Performance Is Attainable 

Fair, 

Acceptable Operator Mental Effort Is 

Mild Difficulty 

Required To Attain Adequate System 


Performance 



Minor But Annoying 
Difficulty 

Moderately High Operator Mental Effort Is 
Required To Attain Adequate System 
Performance 



Moderately Objectionable 
Difficulty 

High Operator Mental Effort Is Required To 
Attain Adequate System Performance 


Very Objectionable But 
Tolerable Difficulty 

MaximumOperator Mental Effort Is Required 
To Attain Adequate System Performance 



Major Difficulty 

Maximum Operator Mental Effort Is Required 
To Bring Errors to Moderate Level 

Major Difficulty 

Maximum Operator Mental Effort Is Required 
To Avoid Large or Numerous Errors 

Major Difficulty 

Intense Operator Mental Effort Is Required 
To Accomplish Task, But Frequent Or 
Numerous Errors Persist 


ir: 

Instructed Task Cannot 
Be Accomplished Reliably 




Operator Decisions 


4. Follow the flow chart above to select the peak workload you experienced during the 
scenario you just completed. 

> Rating of your peak workload level: 

> Please select the segment of flight during which your peak workload level 
occurred (you may select more than one segment if appropriate): 

o >1 8,000ft (cruise, initial descent) 
o 1 8,000ft-l 1 ,000ft (descent, approach check) 
o 1 l,000ft-5, 000ft (TRACON, low altitude merge) 
o <5, 000ft (final approach, configure aircraft) 

(Optional) Use the space provided below to record any clarifying comments or interesting 
observations related to the workload level you experienced during the last run: 
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5. Respond to each of the statements shown below using a scale ranging from “1” 

(Completely Disagree) to “7” (Completely Agree). Circle one number in conjunction 
with each statement. 


Rating Scale 

1 2 3 4 5 6 7 


Completely Completely 

Disagree Agree 


Relevant information, including operational 
plans, decisions, and changes in aircraft state, 
were effectively communicated between 
yourself and your crewmember. 

1 

2 

3 

4 

5 

6 

7 

The time available for tasks was well 
managed. 

1 

2 

3 

4 

5 

6 

7 

The IM commanded speeds were 
operationally acceptable and appropriate. 

1 

2 

3 

4 

5 

6 

7 

The frequency of the IM speed commands 
was acceptable at all times throughout the 
scenario. 

1 

2 

3 

4 

5 

6 

7 

I understood why the IM commanded speeds 
were provided (i.e. the IM commanded 
speeds made sense). 

1 

2 

3 

4 

5 

6 

7 

The use of voice communications to provide 
the IM clearance(s) was acceptable in this 
scenario. 

1 

2 

3 

4 

5 

6 

7 

The amount of head down time required to 
input information from the IM clearance(s) 
into the EFB was acceptable. 

1 

2 

3 

4 

5 

6 

7 

During this scenario, entering IM clearance 
information into the EFB was easy and 
intuitive. 

1 

2 

3 

4 

5 

6 

7 

During this scenario, it was easy to obtain 
needed information from the IM displays 

1 

2 

3 

4 

5 

6 

7 

The flight crew procedures for the events in 
this scenario were complete and acceptable. 

1 

2 

3 

4 

5 

6 

7 
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6. Please answer the following questions regarding the IM procedures you were asked to 
exercise in the scenario that you just completed. 

a. Did the procedures contain missing steps? 

YES 

NO 

b. Did the procedures contain extra steps that were unnecessary? 

YES 

NO 

c. Were the procedural steps logical and easy to follow? 

YES 

NO 

7. Please briefly explain any undesirable ratings from the statements above: 


8. Describe any unusual or unexpected event(s) and your reaction(s), if applicable: 


9. (Optional) This space is reserved for any additional comments related to awareness and 
acceptability issues. If you have any clarifying comments or interesting observations 
related to awareness and acceptability issues, please provide them below: 
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Appendix D. Post-Experiment Questionnaire 

Pilot participants were asked to complete the following post-experiment questionnaire prior to 
participating in a group debrief session with the research team: 


This questionnaire is intended to gather your comments and suggestions regarding the 
experiment itself as well as the Interval Management concept 

This questionnaire contains items associated with each of the following categories: 

• Simulator and Flight Scenarios 

• Training 

• Interval Management Procedures 

• Interval Management Displays 

• Spacing Tool 

• Additional Comments 


Simulator and Flight Scenarios 

1. Was the workload required to operate the simulator much less than, the same as, or 
greater than the workload required to fly an actual aircraft? 


Much 

Moderately 

Slightly 

The 

Slightly 

Moderately 

Much 

More 

More 

More 

Same 

Less 

Less 

Less 

1 

2 

3 

4 

5 

6 

7 


Please provide any additional comments regarding the simulator: 


2. Please share your impressions of the flight scenarios (e.g., comment on their level of 
realism, appropriateness, and/or diversity) and comment on how the design of the 
scenarios impacted your ability to perform the spacing task: 


Training 

3. Did you receive adequate training with respect to flying the simulator? 

YES 

NO 

If not, briefly describe how simulator training can be improved: 
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4. Did you receive adequate training with respect to the IM spacing procedure and the 
spacing tool? 

YES 

NO 

If not, briefly describe how IM procedure or spacing tool training can be improved: 

5. Did you receive adequate training with respect to the entry and interpretation of 
information presented on the EFB? 

YES 

NO 

If not, briefly describe how EFB training can be improved: 

6. Did you receive adequate training with respect to the interpretation of information 
presented on the AGD? 

YES 

NO 

If not, briefly describe how AGD training can be improved: 


Interval Management Procedures 

7. Within this experiment four different aspects of the IM procedures were tested: 
terminating the IM operation, suspending and then resuming the IM operation, 
amending the IM spacing goal, and the loss of the lead aircraft’s ADS-B, as well as 
nominal operations. The following questions are intended to gather your feedback 
about the procedures used for each aspect of the IM operation that was tested (note that 
the last question asks about the general IM procedures). 

a) Were the procedures for terminating IM operation complete, accurate, and logical? 

YES 

NO 

Please provide any suggestions regarding the way(s) in which the procedures for 
terminating the IM operation may be improved: 
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b) Were the procedures for suspending and then resuming an IM operation complete, 
accurate, and logical? 

YES 

NO 

Please provide any suggestions regarding the way(s) in which the procedures for 
suspending the IM operation may be improved: 

c) Were the procedures for amending the IM spacing goal complete, accurate, and 
logical? 

YES 

NO 

Please provide any suggestions regarding the way(s) in which the procedures for 
amending the IM spacing goal may be improved: 

d) Were the procedures for reacting to the loss of your lead aircraft’s ADS-B signal 
complete, accurate, and logical? 

YES 

NO 

Please provide any suggestions regarding the way(s) in which the procedures for 
reacting to the loss of your lead aircraft’s ADS-B signal may be improved: 

e) Were the general (nominal) IM procedures complete, accurate, and logical? 

YES 

NO 

Please provide any suggestions regarding the way(s) in which the general IM 
procedures may be improved: 

8. Was the IM phraseology used in this experiment correct and intuitive? 

YES 

NO 

If “no,” why not, and what could be done to improve the phraseology? 
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9. How difficult do you think it would be for a typical flight crew to learn and integrate 
the IM spacing procedures into their current daily operational flight procedures? 


Very 

Moderately 

Slightly 

Slightly 

Moderately 

Very 

Difficult 

Difficult 

Difficult Neutral 

Easy 

Easy 

Easy 

1 

2 

3 4 

5 

6 

7 


Briefly describe any challenges involved with integrating the IM procedures with 
existing procedures: 


10. Do you think the division of tasks between the Pilot Flying (PF) and Pilot Monitoring 
(PM) was both desirable and fit within the current distribution of tasks between PF and 


PM? 

YES 

NO 


If “no,” what was wrong with the division, and how would you reallocate the tasks? 


11. Given the experience with IM that you gained during this simulation, what is your 
overall assessment of the safety of the spacing procedure compared with current day 
operations? (“Safety” in this question refers to your holistic opinion to include 
workload, awareness, position relative to other aircraft, etc.) 


Not Safe 

Moderately 

Slightly 


Slightly 

More 

Moderately 

Much 

More 

At All 

Less Safe 

Less Safe 

As Safe 

Safe 

More Safe 

Safe 

1 

2 

3 

4 

5 

6 

7 


Briefly describe any characteristic or event that determined your rating (if 
appropriate): 


Interval Management Displays 

12. In general, did you find the process of entering IM clearance information into the EFB 
easy and intuitive? 


YES 

NO 


If “no,” what can be done to improve the process of loading information into the EFB? 
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13. Did the AGD and EFB provide you with the information you needed/desired to safely 
and correctly conduct IM, and was this information easy to obtain when needed? 

YES 

NO 

If “no,” what information was missing, or how can the information be presented better? 


Function Allocation and the Spacing Tool 

14. Did following the IM commanded speed and procedure ever cause unexpected or 
undesirable behavior? 

YES 

NO 

If “yes,” please explain what the unexpected or undesirable behavior was: 


15. Did you find the responsibility of using onboard automation to achieve a spacing 
interval behind a lead aircraft acceptable (when ATC is responsible for separation)? 

YES 

NO 

If “no,” why not, and what could be done to make the responsibility or workload 
acceptable? 


16. Did you find your level of engagement with the IM automation acceptable (i.e. the level 
of decision making ability you had, and your understanding of the reasoning behind IM 
speeds that were commanded)? 

YES 

NO 

Please explain your answer. 


Additional Comments 

17. Do you have any additional comments about the experiment? 
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Appendix E. Example Interval Management (IM) Clearance and 
Related Use of and Display of Information on the 
Prototype Flight Deck-based Interval Management 
(FIM) Crew Interface 

The initial (empty) page of the Interval Management (IM) application residing on the electronic 
flight bag (EFB) portion of the prototype Flight Deck-based Interval Management (FIM) crew 
interface used during the Interval Management for Near-term Operations Validation of 
Acceptability (IM-NOVA) experiment is shown in Figure E-l. 


Command Speed 
Display Area 


Fast / Slow Indicator 
Display Area 



Button used to access 
assigned spacing goal 
data entry page 

Status Box 

- Button used to access 
target aircraft callsign 
data entry page 

- Button used to access 
target aircraft route 
data entry page 


Situation Display 
Note : Ownship aircraft is 
shown using a white chevron 
symbol. In-range ADS -B 
traffic aircraft are shown us ine 
cyan chevron symbols, with 
relative altitude and 
c 1 imb/de s cend arrows shown 
as appropriate 


Zoom In / Out Buttons 


Figure E-1. Initial page of the IM application residing on the prototype FIM crew 

interface’s EFB. 

Figure E- 2-Figure E-l 1 illustrate the use of the prototype FIM crew interface during IM 
clearance data entry activities as well as the activation, suspension, resumption, and 
cancellation/termination of FIM operations. Below is an example of an IM clearance issued via 
voice communications to a flight crew by air traffic control (ATC): 
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NASA 3, for Interval Spacing, cross JIFFY at 1432 plus 30 sec, when able space 90 sec 
behind Delta Alpha Lima 877 on Bonham 5, Fort Smith Transition. Report commencing 
interval spacing. 5 

As shown in Figure E-2, a three-step procedure is used to enter information associated with the 
IM achieve-by point (i.e., JIFFY) into the IM application that resides on the EFB. First, the “IM 
WPT” button is pressed. This action brings up a page that displays all the published waypoints 
on the arrival, allowing the flight crew to press a button associated with “JIFFY.” Lastly, the 
“ENTER” button is pressed. 



5 Note that the IM clearance was designed to provide the flight crew with all required information in an expected 
format. The target aircraft is identified using its three letter phonetic callsign, which corresponds with the callsign 
included in the target aircraft callsign list presented on the EFB (as shown in Figure E-5). The three letter identifier 
was used to assist with the identification on lesser known callsigns. 
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Figure E-3 shows the two-step procedure associated with inputting the scheduled time of arrival 
(STA) at the IM achieve-by point (i.e., 1432 plus 30 sec) into the EFB. First, the “IM STA” 
button is pressed. Then, the touch screen’s keypad is used to enter required data into a 
“scratchpad” area, and the “ENTER” button is pressed. 



Step 1 Step 2 

Figure E-3. Two-step procedure for inputting STA at the IM achieve-by point 
information into the prototype FIM crew interface’s EFB. 
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Figure E-4 shows the two-step procedure associated with inputting the assigned spacing goal 
(ASG) (i.e., 90 sec) information into the EFB. First, the “IM GOAL” button is pressed. Then, the 
touch screen’s keypad is used to enter required data into a “scratchpad” area, and the “ENTER” 
button is pressed. 




Step 1 Step 2 

Figure E-4. Two-step procedure for inputting assigned spacing goal information 

into the prototype FIM crew interface’s EFB. 
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Figure E-5 shows the three-step procedure associated with inputting a target aircraft’s callsign 
(i.e., Delta Alpha Lima 877) into the EFB. First, the “TGT ACFT” button is pressed. When the 
target aircraft is within Automatic Display Surveillance-Broadcast (ADS-B) range, its callsign 
can be selected, with a button press, from a selectable list. Lastly, the “ENTER” button is 
pressed. 
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Figure E-5. Three-step procedure for inputting a target aircraft’s callsign into the 

prototype FIM crew interface’s EFB. 
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Figure E-6 shows the three-step procedure associated with inputting a target aircraft’s arrival and 
transition (i.e., Bonham 5, Fort Smith Transition) information into the EFB. First, the “TGT 
RTE” button is pressed. Next, the target aircraft’s arrival is selected, with a button press, from a 
selectable list. Then, the target aircraft’s transition is selected, with a button press, from a 
selectable list, and the “ENTER” button is pressed. 
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Figure E-6. Three-step procedure for inputting a target aircraft’s arrival and 
transition information into the prototype FIM crew interface’s EFB. 
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After entering the achieve-by point, STA, ASG, target aircraft callsign, and target aircraft route 
information into the EFB, the flight crew can activate the IM clearance by pressing the 
“ACTIVATE” button located in the lower right portion of the EFB’s touch screen (shown in 
Figure E-7). Once pressed, the “ACTIVATE” button changes to a “SUSPEND” button, and the 
IM system indicates that it is performing various calculations by displaying the word 
“CALCULATING” in the EFB’s status box (shown in Figure E-7). 




Figure E-7. IM clearance activation. 
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The flight crew flies in STA mode until ADS-B information from their target aircraft is acquired 
and FIM operations begin. As shown in Figure E-8, an indication of STA mode is displayed in 
the EFB’s status box, and a “CMD SPD,” in Mach or knots, and a “FAST/SLOW” indication, in 
knots, are displayed on both the EFB and ADS-B guidance display (AGD) components of the 
prototype FIM crew interface. 



EFB 


AGD 


Figure E-8. Information elements associated with STA mode displayed on the 
EFB and AGD components of the prototype FIM crew interface. 
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When ADS-B information from the target aircraft is acquired and FIM operations are underway, 
the EFB displays the target aircraft’s callsign, a commanded speed, and a “FAST/SLOW” 
indication (as shown in Figure E-9). Additionally, as shown in Figure E-9, the AGD supplies a 
white “IM” light located in the upper left corner of the device that indicates that the FIM aircraft 
is actively spacing relative to a target aircraft, and the AGD repeats the commanded speed and 
“FAST/SLOW” indications presented on the EFB. 



EFB 


AGD 


Figure E-9. Information elements associated with FIM operations displayed on 
the EFB and AGD components of the prototype FIM crew interface 
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Figure E-10 shows the three-step procedure associated with using the EFB to suspend and 
resume FIM operations. First, the “SUSPEND” button is pressed. This causes several things to 
happen: the “SUSPEND” button changes to a “RESUME” button, and a “TERMINATE” button 
appears; the commanded speed (“CMD SPD”) and “FAST/SLOW” indication are removed; and 
the word “SUSPENDED” is displayed in the EFB’s status box. At this point, the flight crew 
follows ATC’s speed commands. When instructed by ATC to resume FIM operations, the flight 
crew presses the “RESUME” button, causing the following things to happen: the “RESUME” 
button changes to a “SUSPEND” button; the “CMD SPD” and “FAST/SLOW” indication are 
presented; and the target aircraft’s callsign is displayed in the EFB’s status box. At this point, the 
flight crew resumes FIM operations and follows IM commanded speeds. 



Step 1 


Step 2 


Step 3 


Figure E-10. Three-step procedure for suspending and then resuming IM 
spacing using the prototype FIM crew interface’s EFB. 
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Figure E-l 1 shows the two-step procedure and resulting data re-entry screen associated with 
using the EFB to terminate FIM operations. First, the “SUSPEND” button is pressed, causing the 
following things to happen: the “SUSPEND” button changes to a “RESUME” button, and a 
“TERMINATE” button appears; the “CMD SPD” and “FAST/SLOW” indication are removed; 
and the word “SUSPENDED” is displayed in the EFB’s status box. Next, the “TERMINATE” 
button is pressed; all IM clearance data are removed from the EFB; and ATC instructions are 
followed. If FIM operations are to take place at this point, information from a new IM clearance 
must be entered into the EFB using the initial (empty) page of the IM application. 



Step 1 Step 2 Initial page of 

IM application 

Figure E-11. Two-step procedure, and resulting data (re-)entry screen, associated 
with terminating IM spacing using the prototype FIM crew interface’s 

EFB. 
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